Taming Uncertainty in the Assurance Process of Self-Adaptive Systems: a Goal-Oriented Approach

Goals are first-class entities in a self-adaptive system (SAS) as they guide the self-adaptation. A SAS often operates in dynamic and partially unknown environments, which cause uncertainty that the SAS has to address to achieve its goals. Moreover, besides the environment, other classes of uncertainty have been identified. However, these various classes and their sources are not systematically addressed by current approaches throughout the life cycle of the SAS. In general, uncertainty typically makes the assurance provision of SAS goals exclusively at design time not viable. This calls for an assurance process that spans the whole life cycle of the SAS. In this work, we propose a goal-oriented assurance process that supports taming different sources (within different classes) of uncertainty from defining the goals at design time to performing self-adaptation at runtime. Based on a goal model augmented with uncertainty annotations, we automatically generate parametric symbolic formulae with parameterized uncertainties at design time using symbolic model checking. These formulae and the goal model guide the synthesis of adaptation policies by engineers. At runtime, the generated formulae are evaluated to resolve the uncertainty and to steer the self-adaptation using the policies. In this paper, we focus on reliability and cost properties, for which we evaluate our approach on the Body Sensor Network (BSN) implemented in OpenDaVINCI. The results of the validation are promising and show that our approach is able to systematically tame multiple classes of uncertainty, and that it is effective and efficient in providing assurances for the goals of self-adaptive systems

A Property Specification Pattern Catalog for Real-Time System Verification with UPPAAL

Context: The goal of specification pattern catalogs for real-time requirements is to mask the complexity of specifying such requirements in a timed temporal logic for verification. For this purpose, they provide frontends to express and translate pattern-based natural language requirements to formulae in a suitable logic. However, the widely used real-time model checking tool UPPAAL only supports a restricted subset of those formulae that focus only on basic and non-nested reachability, safety, and liveness properties. This restriction renders many specification patterns inapplicable. As a workaround, timed observer automata need to be constructed manually to express sophisticated requirements envisioned by these patterns. Objective: In this work, we fill these gaps by providing a comprehensive specification pattern catalog for UPPAAL. The catalog supports qualitative and real-time requirements and covers all corresponding patterns of existing catalogs. Method: The catalog we propose is integrated with UPPAAL. It supports the specification of qualitative and real-time requirements using patterns and provides an automated generator that translates these requirements to observer automata and TCTL formulae. The resulting artifacts are used for verifying systems in UPPAAL. Thus, our catalog enables an automated end-to-end verification process for UPPAAL based on property specification patterns and observer automata. Results: We evaluate our catalog on three UPPAAL system models reported in the literature and mostly applied in an industrial setting. As a result, not only the reproducibility of the related UPPAAL models was possible, but also the validation of an automated, seamless, and accurate pattern- and observer-based verification process. Conclusion: The proposed property specification pattern catalog for UPPAAL enables practitioners to specify qualitative and real-time requirements...Comment: Accepted Manuscrip

Runtime Verification of Self-Adaptive Systems with Changing Requirements

To accurately make adaptation decisions, a self-adaptive system needs precise means to analyze itself at runtime. To this end, runtime verification can be used in the feedback loop to check that the managed system satisfies its requirements formalized as temporal-logic properties. These requirements, however, may change due to system evolution or uncertainty in the environment, managed system, and requirements themselves. Thus, the properties under investigation by the runtime verification have to be dynamically adapted to represent the changing requirements while preserving the knowledge about requirements satisfaction gathered thus far, all with minimal latency. To address this need, we present a runtime verification approach for self-adaptive systems with changing requirements. Our approach uses property specification patterns to automatically obtain automata with precise semantics that are the basis for runtime verification. The automata can be safely adapted during runtime verification while preserving intermediate verification results to seamlessly reflect requirement changes and enable continuous verification. We evaluate our approach on an Arduino prototype of the Body Sensor Network and the Timescales benchmark. Results show that our approach is over five times faster than the typical approach of redeploying and restarting runtime monitors to reflect requirements changes, while improving the system's trustworthiness by avoiding interruptions of verification.Comment: 18th Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2023

A Modeling Strategy for the Verification of Context-Oriented Chatbot Conversational Flows via Model Checking

Verification of chatbot conversational flows is paramount to capturing and understanding chatbot behavior and predicting problems that would cause the entire flow to be restructured from scratch. The literature on chatbot testing is scarce, and the few works that approach this subject do not focus on verifying the communication sequences in tandem with the functional requirements of the conversational flow itself. However, covering all possible conversational flows of context-oriented chatbots through testing is not feasible in practice given the many ramifications that should be covered by test cases. Alternatively, model checking provides a model-based verification in a mathematically precise and unambiguous manner. Moreover, it can anticipate design flaws early in the software design phase that could lead to incompleteness, ambiguities, and inconsistencies. We postulate that finding design flaws in chatbot conversational flows via model checking early in the design phase may overcome quite a few verification gaps that are not feasible via current testing techniques for context-oriented chatbot conversational flows. Therefore, in this work, we propose a modeling strategy to design and verify chatbot conversational flows via the Uppaal model checking tool. Our strategy is materialized in the form of templates and a mapping of chatbot elements into Uppaal elements. To evaluate this strategy, we invited a few chatbot developers with different levels of expertise. The feedback from the participants revealed that the strategy is a great ally in the phases of conversational prototyping and design, as well as helping to refine requirements and revealing branching logic that can be reused in the implementation phase

GODA: A goal-oriented requirements engineering framework for runtime dependability analysis

Many modern software systems must deal with changes and uncertainty. Traditional dependability requirements engineering is not equipped for this since it assumes that the context in which a system operates be stable and deterministic, which often leads to failures and recurrent corrective maintenance. The Contextual Goal Model (CGM), a requirements model that proposes the idea of context-dependent goal fulfillment, mitigates the problem by relating alternative strategies for achieving goals to the space of context changes. Additionally, the Runtime Goal Model (RGM) adds behavioral constraints to the fulfillment of goals that may be checked against system execution traces. Objective: This paper proposes GODA (Goal-Oriented Dependability Analysis) and its supporting framework as concrete means for reasoning about the dependability requirements of systems that operate in dynamic contexts. Method: GODA blends the power of CGM, RGM and probabilistic model checking to provide a formal requirements specification and verification solution. At design time, it can help with design and implementation decisions; at runtime it helps the system self-adapt by analyzing the different alternatives and selecting the one with the highest probability for the system to be dependable. GODA is integrated into TAO4ME, a state-of-the-art tool for goal modeling and analysis. Results: GODA has been evaluated against feasibility and scalability on Mobee: a real-life software system that allows people to share live and updated information about public transportation via mobile devices, and on larger goal models. GODA can verify, at runtime, up to two thousand leaf-tasks in less than 35ms, and requires less than 240 KB of memory. Conclusion: Presented results show GODA's design-time and runtime verification capabilities, even under limited computational resources, and the scalability of the proposed solution

An architecture for mission coordination of heterogeneous robots

Context: Robots can potentially collaborate to execute a variety of tasks in the service robots domain. However, developing applications of service robots can be complex due to the high level of uncertainty and required level of autonomy. Objective: We aim at contributing an architecture for the development of applications, capable of coordinating multi-robot missions, and that promotes modifiability and seamless integration of independently developed components. Method: In this work, we introduce MissionControl: an ensemble-based architecture to coordinate missions of heterogeneous robots to autonomously form coalitions. MissionControl comprises a component model and a runtime environment. The component model specifies how the system can be extended for different robot’s behaviors and environments. The runtime environment provides the processes required for coordinating the execution of missions at runtime. Results: We evaluated MissionControl in a simulated environment in the healthcare domain. We randomly generated 81 scenarios with uncertainty in the robots’ initial configurations. Then, each scenario was executed 8 times (i.e. 648 runs), where we evaluated the feasibility and efficiency of MissionControl for autonomously forming coalitions against a baseline approach that uses a random robot allocation. Statistical hypotheses testing yielded that MissionControl was able to achieve higher success rates while reducing the required time to conclude a mission, when compared to a baseline approach. We also perform an evaluation of the key quality attributes of the architecture, i.e. modifiability and integrability. Conclusions: MissionControl demonstrated itself able to coordinate multi-robot missions by autonomously assigning missions. Despite the error-prone robotic mission environment and demanding computational resources, MissionControl led to a significant increase in the success rate, while also decreasing the time required to conclude robotic missions when compared to a baseline approach